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1 Linear Algebra review 


1.1 Matrix multiplication, and why we care 
Suppose we have a system of equations over some field F, e.g. 
321 + v2 — 8x3 => 
9x1 —%y-%3 = 2 


The set of ordered triples (1, 22,273) that solve the system is precisely the set of 3-element vectors x € F? that 
solve the matrix equation 
1 3 1 -8 
cas | 2 9 -1 -1 | 
Suppose that A is a square matrix. The equation Ax = 0 always has a solution (all zeros). An interesting 
question is to study when there are no more solutions. 


| where A =| 


Definition 1 A square matriz A is nonsingular if Ax = 0 has only one solution: the all-zeros vector. 
Often, nonsingular matrices are also called invertible, for the following reason. 

Theorem 1 The following are equivalent: 

(i) The square matriz A is nonsingular. 

(ii) There exists another matrix, denoted A~' such that AW‘A =I = AA}. 
Solution: 


e (i) to (ii) row-reduction, find that A always has RREF = identity 

e then can always solve Ax = (anything) 

e so can for example solve Ax = I, by doing it column by column. 

e now we know there is some B such that AB = I. 

e also, the row operations themselves were some left multiplication of matrices 
e so we also have some C so that CA = I. 


e Then B=IB=CAB=CI=C, so they are the same. 


(ii) to (i): just multiply by inverse. 
For non-square matrices, the most important fact is the following: 


Theorem 2 If A has more columns than rows, then the equation Ax = 0 has more solutions than just the 
all-zeros vector. 


Solution: RREF, run out of rows before columns, so we don’t have the sentinel 1 in every column. This 
enables us to choose arbitrary nonzero values for all non-sentinel columns, and then still read off a valid solution 
by backtracking the sentinel columns. 


1.2. Vectors 


A vector is typically represented as an arrow, and this notion is widely used in Physics, e.g., for force, velocity, 
acceleration, etc. In Mathematics, vectors are treated more abstractly. In this lecture, we will mostly use 
concrete vectors, which one can think of as columns of numbers (the coordinates). The fundamental operation 
in Linear Algebra is the linear combination. It is called “linear” because vectors are not multiplied against each 
other. 


Definition 2 Given vectors v1,...,Vz, and real coefficients c1,...,ch, the sum civ, +++: + Crvp is called a 
linear combination. 


It’s intuitive that two vectors “usually” do not point in the same direction, and that three vectors “usually” 
do not all lie in the same plane. One can express both of the previous situations as follows: 


Definition 3 Let {v1,...,v,} be a collection of vectors. One says that they are linearly dependent if: 
e One of the vectors can be expressed as a linear combination of the others. 


e There is a solution to cyv, +++: +cpvyp = 0, using real numbers cy,...,Ccxh, where not all of the c;’s are 
zero. 


Solution: 
e (i) to (ii): swap signs on the linear combination. 
e (ii) to (i): pick a nonzero, then shift all others to other side, and divide by the nonzero coefficient. 


A fundamental theorem in Linear Algebra establishes that it is impossible to have “too many” linearly 
independent vectors. This is at the core of many Combinatorial applications. 


Theorem 3 The maximum possible number of linearly independent vectors in R” is n. 


Solution: Suppose we have more than n, say v1,...,Vn+1- Try to solve for the coefficients. This sets up 
a matrix equation with more columns than rows, which by previous we know to have a nontrivial solution. 


2 Combinatorics of sets 
We begin with a technical lemma. 


Lemma 1 Let A be a square matrix over R, for which all non-diagonal entries are all equal to some t > 0, and 
all diagonal entries are strictly greater than t. Then A is nonsingular. 


Proof. Ift = 0, this is trivial. Now suppose t > 0. Let J be the all-ones square matrix, and let D = A—tJ. Note 
that D is nonzero only on the diagonal, and in fact strictly positive there. We would like to solve (tJ + D)x = 0, 
which is equivalent to Dx = —tJx. Let s be the sum of all elements in x, and let the diagonal entries of D 
be di,...,dn, in order. Then, we have djx; = —ts > a; = —(t/d;)s. But since t and d; are both strictly 
postive, this forces every x; to have opposite sign from s, which is impossible unless all x; = 0. Therefore, A is 
nonsingular. 

Solution: ALTERNATE: 

Let J be the all-ones square matrix, and let D = A —tJ. Note that D is nonzero only on the diagonal, and 
in fact strictly positive there, so it is a positive definite matrix. Also, J is well-known to be positive semidefinite 
(easy to verify by hand), so A is positive definite. In particular, this means that x’ Ax = 0 only if x = 0, 
implying that Ax = 0 only for x = O. This is equivalent to A being nonsingular. 


Now try the following problems. The last two come from 102 Combinatorial Problems, by T. Andreescu 
and Z. Feng. 


1. (A result of Bourbaki on finite geometries; also appeared in St. Petersburg Olympiad.) Let X be a finite 
set, and let F be a family of distinct proper subsets of X. Suppose that for every pair of distinct elements 
in X, there is a unique member of F which contains both elements. Prove that |F| > |X|. 


Solution: Let X = [n] and F = {Aj,...,Am}. We need to show that n < m. Define the m x n 
incidence matrix A over R by putting 1 in the i-th row and j-th column if 7 € A;. Consider the product 
AA, which is an n x n matrix. For i ¥ j, its entry at (i, 7) is precisely 1. 


Also, the diagonal entries are strictly larger than 1, because if some element 7 € X belongs to only one 
set Ay € F, then the condition implies that every element i € X is also in A,, contradicting requirement 
that A; be proper. 


Therefore, A? A is nonsingular by Lemma 1. But if A has more rows than columns, then it would have 
some x # 0 such that Ax = 0, hence AT Ax = 0. Therefore, A doesn’t have more rows than columns, 
Le, n<m. 


2. (Fisher’s inequality) Let C = {Aj,...,A,} be a collection of distinct subsets of {1,...,n} such that every 
pairwise intersection A; A; (i 4 j) has size t, where t is some fixed integer between 1 and n inclusive. 
Prove that |C| <n. 


Solution: Consider the n x r matrix A, where the i-th column of A is the characteristic vector of A;. 
Then, A’ A is ar x r matrix, all of whose off-diagonal entries are t. We claim that the diagonal entries 
are all > t. Indeed, if there were some |A;| which were exactly t, then the structure of C must look like 
a “flower,” with one set A; of size t, and all other sets fully containing A; and disjointly partitioning the 
elements of [n] \ A; among them. Any such construction has size at most 1+ (n — t) < n, so we would 
already be done. 


Therefore, A? A is nonsingular by Lemma 1, and the previous argument again gives r < n. 


3. Let Aj,...,A, be a collection of distinct subsets of {1,...,n} such that all |A;| are even, and also all 
|A; A,| are even for i 4 7. How big can r be, in terms of n? 


Solution: Arbitrarily cluster [n] into pairs, possibly with one element left over. Then take all possible 
subsets where we never separate the pairs; this gives r up to 2/2], 


But this is also best possible. Suppose that S is the set of characteristic vectors of the sets in the extremal 
example. The condition translates into S being self-orthogonal. But S L S => (S) 1 (S$), so extremality 
implies that S is in fact an entire linear subspace, which is self-orthogonal (i.e., S C S+). 


We have the general fact that for any linear subspace, dim $+ = n — dim S. This is because if d = dim S, 
we can pick a basis vj1,...,Uq of S, and write them as the rows of a matrix A. Then, the kernel of A is 
precisely S+, but any kernel has dimension equal to n minus the dimension of the row space (d). 


Therefore, S C S+ implies that dimS < dimS+ = n— dimS, which forces dimS < |n/2]|, so we are 
done. 


4, What happens in the previous problem if we instead require that all |A;| are odd? We still maintain that 
all |A; A,| are even for i F j. 


Solution: Answer: r <n. Work over F2. The characteristic vectors v; of the A; are orthonormal!, so 
they are linearly independent: given any dependence relation of the form >> c;v; = 0, we can dot product 
both sides with v, and conclude that cz, = 0. Thus, there can only be < n of them. 

ALTERNATE: Let A be the n x r matrix where the columns are the characteristic vectors of the Aj. 


Then A’ A equals the r x r identity matrix, which is of course of full rank r. Thus r = rank(A7A) < 
rank(A) <n. 


5. Prove that if all codegrees? in a graph on n vertices are odd, then n is also odd. 


1Strictly speaking, this is not true, because there is no positive definite inner product over Fz. However, if one carries out the 
typical proof that orthonormality implies linear independence, it still works with the mod-2 dot product. 
?The codegree of a pair of vertices is the number of vertices that are adjacent to both of them. 


Solution: First we show that all degrees are even. Let v be an arbitrary vertex. All vertices w € N(v) 
have odd codegree with v, which means they all have odd degree in the graph induced by N(v). Since 
the number of odd-degree vertices in any graph must always be even, we immediately find that |N(v)| is 
even, as desired. 


Let A be the adjacency matrix. Then A? A = J — J. But consider right-multiplying by 1. Al = 0 > 
A’ Al = 0 and J1 = 1, so we need to have J1 = 1, which implies that n is odd. 


ALTERNATE ENDING: Now, let S = {1,v1,...,Un} be the set of n +1 vectors in F} where 1 is the 
all-ones vector and v; is the characteristic vector of the neighborhood of the i-th vertex. There must be 
some nontrivial linear dependence b1 + >>, a;v; = 0. But note that if we take the inner product of this 
equation with vz, we obtain }),,, a; = 0 because 1- vj, = 0 = vg- vy, and v;- vp = 1 for i # k. Hence all 
the a; are equal. Yet if they are all zero, then b is also forced to be zero, contradicting the nontriviality 
of this linear combination. Therefore, all a; are 1, and the equation 5°, Zh Ui = 0 forces n — 1 to be even, 
and n to be odd. 


. (Introductory Problem 38) There are 2n people at a party. Each person has an even number of friends at 
the party. (Here, friendship is a mutual relationship.) Prove that there are two people who have an even 
number of common friends at the party. 


Solution: Let A be adjacency matrix. Suppose for contradiction that every pair of people has an odd 
number of common friends. Then over Fz, we have A’ A = J — I, where J is the all-ones matrix and I 
is the identity. Since all degrees even, Al = 0. Hence A7A1 = 0. But J1 = O because J is a 2n x 2n 
matrix, and J1 = 1. Thus we have 0 = A? Al = (J — I)1 = 1, contradiction. 


. (Advanced Problem 49) A set T is called even if it has an even number of elements. Let n be a positive 
even integer, and let 51,...,S, be even subsets of the set {1,...,n}. Prove that there exist some i 4 j 
such that S$; S; is even. 


Solution: Let A benxn matrix over F> with columns that are the characteristic vectors of the $;. Then 
A’ A = J—I, but Ais singular because A71 = 0. Square the equation. We have (J—I)(J—I) = J?-2J+I 
since J, J commute. But n is even, and we are in Fg, so it is just J, and we get A? AA? A = I. Contradicts 
singularity of A. 


(Uses A nonsingular implies A? nonsingular. Indeed, we need (AB)? = B” A’. So, in particular, if A 
had inverse B, then we have a matrix B? such that it is left inverse of A’. In particular, whenever we go 
to solve A? x = 0, we can left-multiply by B’, and get x = B’0 = 0, so no nontrivial solutions.) 
ALTERNATE: Singularity implies det A7 A = (det A)? = 0. However, det(J—T) is precisely the parity of 
D,,, the number of derangements of [n]. It remains to prove that for even n, D, is odd. But this follows 
from the well-known recursion D, = (n — 1)(Dn-1 + Dn—2), which can be verified by looking at where 
the element n is permuted to. 


Bonus problems (not all linear algebra) 


. (Caratheodory.) A convex combination of points x; is defined as a linear combination of the form )°; a;2%;, 
where the a; are non-negative coefficients which sum to 1. 


Let X be a finite set of points in R%, and let cvx(X) denote the set of points in the convex hull of X, ice., 
all points expressible as convex combinations of the 7; € X. Show that each point x € cvx(X) can in fact 
be expressed as a convex combination of only d+ 1 points of X. 


Solution: Given a convex combination with d+ 2 or more nonzero coefficients, find a new vector with 
which to perturb the nonzero coefficients. Specifically, seek 5°, 8:2; = 0 and >>, 8; = 0, which is d+1 
equations, but with d+ 2 variables 6;. So there is a non-trivial solution, and we can use it to reduce 
another a; coefficient to zero. 


. (Radon.) Let A be a set of at least d+ 2 points in R¢. Show that A can be split into two disjoint sets 
A; U Ag such that cvx(A,) and cvx(Ag) intersect. 


Solution: For each point, create an R¢+1-vector v; by adding a “1” as the last coordinate. We have a 
non-trivial dependence because we have at least d+2 vectors in R“*}, say )>, av; = 0. Split A= A; UA» 
by taking A, to be the set of indices i with a; > 0, and Ag to be the rest. 


S- a= S° (=a). 


tE Ay it€ Ao 


By the last coordinate, we have 


Let Z be that sum. Then if we use a;/Z as the coefficients, we get a convex combination from A, via 
the first d coordinates, which equals the convex combination from Az we get by using (—a;)/Z as the 
coefficients. 


. (Helly.) Let C,,C2,...,Cp be convex sets of points in R?, with n > d+1. Suppose that every d+ 1 of 
the sets have a non-empty intersection. Show that all n of the sets have a non-empty intersection. 


Solution: Induction on n. Clearly true for n = d+ 1, so now consider n > d+ 2, and assume true for 
n—1. Then by induction, we can define points a; to be in the intersection of all Cj, 7 4 7. Apply Radon’s 
Lemma to these a;, to get a split of indices AU B. 


Crucially, note that for each i € A and j € B, the point a; is in Cj. So, each i € A gives a; € Nice C;, 
and hence the convex hull of points in A is entirely contained in all Cj, j € B. 


Similarly, the convex hull of points in B is entirely contained in all C;, 7 ¢ A. Yet Radon’s Lemma gave 
intersecting convex hulls, so there is a point in both hulls, ie., in all Cj, 7 € AUB = [n]. 


. (From Peter Winkler.) The 60 MOPpers were divided into 8 teams for Team Contest 1. They were then 
divided into 7 teams for Team Contest 2. Prove that there must be a MOPper for whom the size of her 
team in Contest 2 was strictly larger than the size of her team in Contest 1. 


Solution: In Contest 1, suppose the team breakdown was s; + --- +s 3 = 60. Then in the i-th team, 
with s; people, say that each person did + of the work. Similarly, in Contest 2, account equally for the 


work within each team, giving scores of +. 


However, the total amount of work done by all people in Contest 1 was then exactly 8, and the total 
amount of work done by all people in Contest 2 was exactly 7. So somebody must have done strictly less 


work in Contest 2. That person saw 
1 1 
— < er 


/ 
Ss; Sy 


i.e., the size of that person’s team on Contest 2 was strictly larger than her team size on Contest 1. 

. (MOP 2007/4/K2.) Let S be a set of 10° points in 3-dimensional space. Show that at least 79 distinct 
distances are formed between pairs of points of S. 

Solution: Zarankiewicz counting for the excluded K3,3 in the unit distance graph. This upper-bounds 


the number of edges in each constant-distance graph, and therefore lower-bounds the number of distinct 
distances. 


. (MOP 2007/10/K4.) Let S be a set of 2n points in space, such that no 4 lie in the same plane. Pick any 
n? +1 segments determined by the points. Show that they form at least n (possibly overlapping) triangles. 


Solution: In fact, every 2n-vertex graph with at least n? +1 edges already contains at least n triangles. 
No geometry is needed. 


. (Sperner capacity of cyclic triangle, also Iran 2006.) Let A be a collection of vectors of length n from Z3 
with the property that for any two distinct vectors a,b € A there is some coordinate i such that 6; = a;+1, 
where addition is defined modulo 3. Prove that |A| < 2”. 

Solution: — For each a € A, define the Z3-polynomial f,(x) := [[j_, (zi — a; — 1). Observe that this is 
multilinear. Clearly, for alla 4b € A, fa(b) = 0, and fa(a) 4 0; therefore, the f, are linearly independent, 


and bounded in cardinality by the dimension of the space of multilinear polynomials in n variables, which 
is 2”. 


